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1  Problem  Review 


This  report  begins  by  providing  a  review  of  the  original  problem  definition  as  well  as  the 
stated  objectives  for  the  Phase  I  research  and  development.  Namely: 

•  Investigate  optimal  platform  routing  techniques  and  algorithms  to  optimize 
collection  for  fusion  metric  benefits  while  satisfying  collection  and  de-confliction 
requirements; 

•  Select,  research  and  define  applications  and  appropriate  optimization  techniques 
for  fusion  of  multi-source  data  from  wide-body  and  UAV  on-board  sensors; 

•  Utilize  Measures  of  Performance  (MOPs)  to  determine  fusion  resultant 
improvements; 

•  Design  an  architecture  to  dynamically  utilize  the  MOPs  to  enhance  algorithm 
performance. 

The  significant  growth  in  UAV  platforms  and  payload  capabilities  offer  a  challenge  and 
opportunity  to  realize  the  benefits  of  cooperative  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR).  The  combination  of  both  (1)  wide-body  and  UAV  cooperative 
systems  or  (2)  multi-UAV  (i.e.,  all-UAV)  cooperative  systems  have  positive,  but 
different  benefits  to  improved  ISR.  Among  other  technology  challenges,  the  routing  and 
data  fusion  requirements  related  to  effective  use  of  these  cooperative  platforms  require 
unique  algorithmic  techniques  and  methods  for  evaluation  of  these  innovative 
approaches. 

Accordingly,  our  Phase  I  objectives  were  to: 

•  Define,  design,  and  develop  patterns  for  synergistic  data  fusion  resource 
management  techniques  for  effective,  cooperative  multi-platform  ISR; 

•  Design,  develop,  and  demonstrate  an  extended  use  case  application  of  our  baseline 
approach; 

•  Extend  capabilities  for  combined  geolocation,  tracking,  threat  estimation  and 
routing  for  modern  hostile  air  defense  environments;  and,  ultimately  lay  the 
foundation  for  the  core  fmCortex™  architecture  (see  Figure  1  and  updated  as 
shown  in  Figure  14). 
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Figure  1.  Proposed  Conceptual  fmCortex™  Architecture  (Original). 

To  meet  these  objectives  and  thus  address  the  defined  problem  we  developed  an  approach 
that  focused  on  a  particular  class  of  cooperative  ISR  missions  that  would  benefit  from 
improved  technologies  from  both  platform  routing,  data  fusion  as  well  as  automated 
dynamic  resource  allocation  techniques  and  automated  support  for  inter-platform  multi¬ 
operator  workflow.  The  primary  focus  was  on  the  combination  of  both  cooperative- 
system  wide-body  and  multi-UAV  systems  that  provide  positive  and  complementary 
benefits  to  improved,  cooperative  Intelligence,  Surveillance,  and  Reconnaissance  (ISR). 
Routing  and  data  fusion  requirements  related  to  effective  uses  of  these  cooperative 
platforms  demand  progressive  algorithmic  techniques  and  methods  embedded  in 
theoretically  innovative  approaches  -  fmCortex™ . 


2  Phase  I  Tasking 

An  overarching  objective  of  the  fmCortex™  concept  is  to  provide  AFRL  RIEA,  Rivet 
Joint  (RJ)  and  the  CMS/KAST  program  a  clear  development  path  with  respect  to  our 
fmCortex™  solution  for  use  with  legacy  and  future  applications.  Our  intention  was  to 
address  this  with  strict  and  constant  consideration  of  the  program  office  needs  and 
requirements  with  the  ultimate  end-objectives  of  greatly  improving  operations  tempo,  and 
reducing  program  costs  by  laying  the  foundation  for  the  development  of  a  flexible  and 
modularized  end-to-end  solution  that  could  accommodate  improvements  while  still 
allowing  for  technical  migration  as  RJ  needs  dictate. 

fmCortex™  will  ultimately  include  those  pre-  and  post-processing  issues  critical  for 
successful  fusion  (e.g.,  validation,  standard-product  assessment,  entity/event/alert 
generation  and  re-generation,  dynamic  tasking,  etc.),  dynamic  resource  management,  as 
well  as  dynamic  database  components.  These  will  be  characterized  collectively  as  a  flow 
management  framework  in  which  we  will  ultimately  develop  a  modular  design  that 
accommodates  future  growth  and  continued  research  and  development  —  by  either  our 
team  or  others  in  the  RJ  community  -  via  a  componentized  linearly  sequential  solution. 
Indeed,  fmCortex™  is  purposefully  envisioned  as  having  an  open,  modular,  and 
extensible  architecture  that  specifically  allows  and  encourages  future  development  and 
integration  as  collective  RJ  technical  advances  progress. 
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We  undertook  the  Phase  I  with  a  set  of  eight  specific  tasks.  Figure  2  illustrates  these  tasks 
and  their  nominal  completion  timelines. 


Activity  tome 

Month  1 

Month2 

Month3 

Month4 

Month5 

Month6 

Month? 

Month8 

Months 

Taskl:  Define  specific 
operations  scenario 
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Dynamic  Resource 
Allocation 
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mathematical  techniques 
and  routing/cueing 
algorithms 

Task  5:  Develop  Fusion 
algorithms 

Task  6:  Define  and 
develop  the  flow 
management  core 
architecture  (fmCortexTM) 

Task  7:  Phase  II  protoype 
design  report 

Task  8:  Market  Study- 
Option 
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Figure  1.  Phase  I  Schedule. 

In  the  following  sections  we  describe  our  Phase  I  accomplishment  under  each  of  the  tasks 
as  illustrated  above. 


2.1  Task  1:  Define  a  specific  operational  scenario  with 

EMPHASIS  ON  MISSION  TASKING 


Our  baseline  approach  considered  a  dynamic  adaptation  to  an  existing  ATO  to  deal  with 
contingencies,  pop-up  threats,  etc,  while  simultaneously  finding  the  best  way  to  reassign 
mission-tasking  segments  to  the  entire  multiplatform  suite.  Our  mathematical 
programming-based  capability  to  optimize  the  task-to-platform  reassignments  allowed  a 
solution  to  be  framed  that  keeps  overall  mission  effectiveness  as  its  central  optimization 
criterion,  and  can  in  fact  be  adaptively-implemented  on  a  time-slice  basis,  so  mission 
managers  can  specify  and  adjust  optimization  criteria  in  real-time.  The  Baseline  Fusion 
approach  involved  both  a  “Level  1”  and  “Level  2”  component  addressing  layered 
optimum  emitter  geolocation  approach  and  emitter  threat  estimation  logic.  The  Use 
Case  involved  a  multiple-mission  context  and  our  demonstrated  solution,  while  showing 
proof-of-concept,  was  a  single  solution  at  a  point  of  time  in  the  mission. 
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The  Scenario,  reported  as  a  “Use  Case”  throughout  the  duration  of  our  Phase  I  contract, 
involved  a  combined  mission  for  regional  IADS  type  surveillance  and  a  simultaneous 
Special  Operations  mission  that  were  interwoven  to  create  the  type  of  environment  where 
Data  fusion  (DF)  and  Dynamic  Resource  Management  (DRM)  technologies  would  be 
emphasized.  The  scenario  involved  a  pop-up  emitter  threat  to  a  Special  Operations 
(SpOps)  mission  that  had  to  be  serviced  while  balancing  the  regional  surveillance 
requirement  for  the  IADS. 

Typical  of  the  Cooperative  ISR  problem,  there  are  interdependencies  between  the  DF 
processes  and  the  DRM  resource  and  mission  optimization  techniques.  The  scenario 
geometry  depicted  in  Figure  3  illustrates  the  platform  general  layouts. 


Figure  3.  Use  Case  Geometry. 

In  the  scenario,  an  RJ  and  three  Global  Hawk  (GH)  UAVs  and  an  F-22  Strike  aircraft  are 
on  a  surveillance  mission  in  area  of  interest  AOI  (1);  part  of  the  mission  is  to  over- fly 
region  R2  where  there  are  known  threats.  The  F-22  Strike  aircraft  provide  air  cover  for 
the  sortie  and  the  UAV’s  are  to  provide  specialized  surveillance  of  certain  priority  sub- 
regions  in  R2.  In  a  portion  of  AOI  (1)  labeled  Rl,  two  mobile  SAM  systems  have  been 
conducting  harassing  operations.  Rl  is  defined  by  the  boundaries  of  previously  deployed 
locations  of  the  emitters;  the  emitters  are  “SA- 10-like”.  The  emitters  have  been 
performing  irregular  emission-no-emission  cycling  with  occasional  interspersed 
relocations.  A  synchronous  SOF  mission  is  also  underway,  involving  squads  of  SOF 
warfighters  being  ferried  by  two  AC- 130  gunships  into  a  target  mission  area  to  the 
Northeast  of  area  of  interest  AOI  (2). 
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En-route  the  RJ  notes  one  of  the  SA-10  emitters  popping  up  somewhere  in  R1  (short  on- 
air  time)  based  on  an  AOA  generally  within  the  previously  seen  Red  zone.  However,  the 
RJ-based  and  short-emitter  uptime-based  geolocation  is  not  precise  enough  to  fully 
understand  the  possible  threat  to  the  SOF  platforms.  The  threat  is  sufficiently  dangerous 
to  the  SOF  mission  in  the  AOI  (2)  area  of  operations  that  RJ  issues  a  coded  Alert  to  the 
SOF  platforms  and  a  request  for  UAV  service  to  provide  precision  geolocation  to  better 
understand  the  SOF  threat  (this  is  Alert  1).  In  support  of  this  request,  pairwise  UAV 
geolocation  calculations  are  made  along  possible  UAV  trajectories  that  would  fly  the 
UAV’s  to  tangent  points  on  the  emitter  kill  range  circle.  The  Task  Nomination  Fogic,  as 
one  basis  to  dynamically  assign  a  particular  pair  of  UAV’s  to  the  geolocation  mission, 
uses  these  estimates.  Other  considerations  in  this  optimization  problem  (see  Section  X 
for  the  details)  involve  the  cost  of  degradation  in  the  planned  surveillance  of  R2  and  other 
factors.  A  particular  UAV  pair  is  reassigned  and  begins  flying  the  “tangent” 
trajectories.  The  UAV  pair  computes  a  series  of  geolocation  updates  according  to  the 
emitting/quiet  time  cycles  of  the  emitter.  At  each  calculation,  the  results  are  given  to  a 
threat  estimation  logic  that  computes  “Time  to  Fethal  Engagement  (TTFE)”  for  the  SOF 
platforms  based  on  the  estimated  (fuzzy)  time  that  it  takes  for  the  platforms  to  enter  the 
kill  zone  of  the  SAM.  Because  of  the  uncertainty  in  the  initial  geolocation  calculations, 
one  F-22  is  dynamically  reassigned  to  provide  air  cover  for  the  SOF  platforms  in  case  the 
kill  zone  envelope  estimate  is  possibly  in  error.  TTFE  is  also  based  on  an  estimate  of  the 
current  SOF  platform  locations,  which  adds  an  error  component  to  the  overall  geometry. 
The  SOF  platforms  do  not  radiate  and  so  the  use  of  GPS  location  information  for  them  is 
not  available.  The  overall  functional  and  processing  flow  is  illustrated  in  Figure  4  which 
also  describes  the  multiple  geolocation  algorithms  that  employ  the  sensing  data  from  both 
the  RJ  and  the  UAV’s,  the  dual  multi-UAV  Fevel  1  fusion  geolocation  algorithms,  the 
fuzzy  logic  and  fusion-based  threat  logic  for  Fevel  2/3  fusion,  and  the  robust,  multi¬ 
condition  platform  and  functional  reassignment  logic. 
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Figure  4.  Overall  Functional  and  Processing  Flow. 

2.1.1  Fusion-Based  Geolocation  Calculations 

Phase  (1)  Calculations:  Centralized  Geolocation 

As  was  described  previously,  the  scenario  begins  with  the  RJ  collecting  some  intermittent 
emissions  from  the  harassing  emitter  in  the  R1  region;  these  data  are  used  by  organic  RJ 
sensors  to  provide  a  rough  geolocation.  The  RJ  initializing  an  onboard  Extended  Kalman 
Filter  (EKF)  to  estimate  the  absolute  position  of  the  unknown  emitting  target  does  this 
initial  calculation.  With  the  first  sensed  emission  the  RJ  samples  at  discrete  time 
intervals.  Measurements  fed  to  the  EKF  come  from  an  RJ  angle  bearing  sensor  and 
ranging  sensor.  Both  sensors  are  assumed  to  have  statistical  errors  based  on  zero-mean 
Gaussian  white  noise. 

Based  on  prior  intelligence  about  region  R1  we  initialize  the  filter  states  somewhere 
within  a  broad  initial  R1  region  with  a  large  error  covariance.  The  EKF  algorithm 
onboard  the  RJ  handles  the  irregular  measurement  data  cycles  of  emission-no-emission 
coming  from  the  harassing  emitter.  When  measurement  data  is  available  during  an 
emission  cycle  the  EKF  updates  the  estimated  state  and  covariance.  The  EKF  propagates 
the  last  state  and  covariance  value  forward  at  each  time  step  in  the  no-emission  cycle. 
Both  state  estimates  errors  and  the  error  covariance  increase  over  time  during  cycles  of 
no  emissions  until  a  measurement  is  received. 

It  is  assumed  that  the  RJ  gets  a  few  “looks”  at  the  emitter  over  a  few  emission  cycles,  and 
once  the  EKF  converges,  the  RJ  takes  the  emitter  location  estimate  and  error  covariance 
and  calculates  a  3-sigma  bound  on  the  estimated  emitter’s  geolocation,  as  shown  in 
Figure  5.  Values  must  be  within  a  selectable  threshold  value  with  respect  to  the  total 
region  encompassing  Rl.  A  kill  zone  is  added  on  the  3-sigma  bound  ellipse  based  on  the 
prior  intelligence  of  emitter  types  in  region  Rl.  Phase  (1)  final  calculations  are 
inadequate  for  accurate  strike  geolocation,  yet  critical  for  Phase  (2)  Error  Covariance 
Reduction. 


Figure  5.  3o  Bounds  (Dashed)  on  the  Geolocation  Error  (Solid). 
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Phase  (2)  Calculations:  Propagated  Centralized  Geolocation  -  Error  Covariance 

Reduction 


Once  the  RJ  Task  Nomination  Logic  calculates  the  UAV  trajectories  with  respect  to  the 
kill  zone,  three  time-correlated  positions  along  those  trajectories  are  selected  for  each 
UAV.  These  three  positions  are  taken  along  a  virtual  computed  path  towards  the  emitter 
but  remaining  outside  the  kill  zone  determined  from  the  RJ  solution,  as  shown  in  Figure 
3;  these  are  the  “tangent”  trajectories  mentioned  above.  Using  these  three  future 
positions  for  each  UAV,  the  RJ  computes  the  covariance  of  the  expected  errors  of  the 
geolocation  solution  using  the  UAVs.  Propagated  geolocation  solutions  depend  now  the 
UAV  sensor  geometry  with  respect  to  the  estimated  geolocation  of  the  emitter.  Note  that 
the  location  of  the  emitter  is  required  to  compute  this  covariance.  It  will  be  assumed  that 
the  RJ-provided  initial  geolocation  is  “close  enough”  to  provide  a  fairly  accurate 
covariance  analysis  for  each  of  the  UAV-based  solutions.  Specifically  it  is  assumed  that 
the  RJ  initial  geolocation  errors  only  produce  second-order  errors  in  the  computation  of 
the  forward  time  propagated  UAV  covariance  analysis.  A  total  of  three  solutions, 
containing  emitter  locations  and  associated  error  covariance  matrices  are  determined.  See 
figure  6. 


Figure  6.  3a  Error  Bounds  (dotted)  Reduction  Phase. 
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A  simple  weighting  for  each  error  covariance  matrix  (P)  is  applied  by  using  the  largest  Py 
of  all  three  P  matrices  and  dividing  through  by  Py.  Normalizing  all  three  P  matrices 
provides  weighting  scheme  with  a  small  range  of  values  describing  the  as  the  lowest  error 
covariance  matrix  as  the  “best”  UAV  pair.  This  pair  is  only  for  geolocation  purposes  and 
not  the  overall  case.  A  geolocation  performance  table  (see  Table  1),  based  on  the 
normalized  data  is  created  to  organize  the  results  in  useful  manner  to  pass  on  to  the  Task 
Nomination  Logic. 

Phase  (3)  Calculations:  Fused  Geolocation  Calculations 

After  the  Task  Nomination  Logic  selects  the  particular  UAV  pair  to  precisely  geolocate 
the  emitter  in  region  Rl,  angle/range  to  emitter  calculations  begin  as  they  approach  the 
kill  zone.  Both  UAVs  are  able  to  run  EKF’s  in  a  decentralized  manner  but  one  selected  as 
team  leader.  The  team  leader  initializes  its  EKF  with  the  available  RJ  Phase  (1)  data  and 
if  in  an  emission  cycle,  fuses  both  angle/range  observations.  If  not  within  an  emission 
cycle  the  leader  propagates  the  state  estimate  and  the  error  covariance  forward  in  time. 

Fusion  of  the  estimate  state  takes  place  when  the  RJ  and  the  UAV  team  combine  their 
geolocation  estimates  on  board  the  RJ  (i.e.,  the  UAV  data  are  communicated  to  the  RJ 
throughout  the  mission).  A  Global  EKF  or  fusion  node  based  on  distributed  architecture 
within  the  RJ  is  initialized  once  the  team  of  UAVs  has  sent  their  geolocation  result.  The 
UAV  team  is  now  a  Local  [Extended]  Kalman  Filter  (LKF)  and  the  second  EKF  using 
angle  and  range  measurements  is  also  a  LKF,  the  third  filter  is  the  new  Fusion  Node 
(FN). 

Within  this  distributed  architecture  each  LKF  has  a  sensor  to  input  measurements.  Now 
in  the  FN  the  inputs  are  taken  from  the  output  estimates  of  the  LKF  and  produce  a  new 
estimate  based  on  the  fused  estimate  of  each  LKF.  This  architecture  is  hierarchical  fusion 
without  feedback  to  any  of  the  LKF.  An  optimal  estimate  cannot  be  guaranteed  as  in  the 
centralized  architecture,  but  near  optimal  and  consistent  estimates  of  the  state  should  be 
an  output  from  the  FN. 

As  the  RJ  and  UAV  combine  geolocation  estimates  their  respective  geometries  vary  in 
time  allowing  better  observations  of  the  emitting  target.  However  at  the  FN  the  error 
covariance  may  vary  in  undesirable  manner  when  it  receives  redundant  geolocation 
estimates.  Because  of  this  the  FN  may  become  over  confident,  instead  of  keeping  the 
same  level  of  confidence  for  the  emitter  error  covariance.  Also,  error  covariance 
correlations  cannot  be  optimally  accounted  for  in  the  LKF  or  at  the  FN  unlike  a 
centralized  EKF. 

Applying  Covariance  Intersection  (Cl)  in  the  FN  meets  both  requirements  of  near  optimal 
and  consistent  geolocation  estimates.  At  the  FN  each  LKF  sends  their  estimate  and  error 
covariance  matrix  for  calculations  using  CI.  The  Cl  gives  an  unbiased  estimate  that  is  a 
linear  combination  of  all  information  received.  Noting  that  each  piece  of  information  has 
knowledge  statistical  associated  with  it  we  determine  the  necessary  weights  for  an 
unbiased  estimate.  Each  piece  of  information  is  weighted  so  the  sum  of  the  weights  equal 
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1  and  the  domain  is  [0,  1],  Optimizing  the  weights  we  can  find  optimal  weights  for  N 
amounts  of  information  to  be  fused. 

Once  the  minimal  FN  error  covariance  is  achieved  for  weapons  strike  the  RJ,  sends  the 
geolocation  3-sigma  bounds  and  estimate  to  the  strike  platforms.  Fusion-based 
geolocation  is  a  robust  architecture  compared  to  a  centralized  version.  The  distributed 
architecture  minimizes  single  point  failures  in  geolocation  calculations  while  the  UAV 
team  and  RJ  are  en-route  to  separate  regions  R1  and  R2.  These  operations  are  depicted  in 
Figure  7  (LKF  =  Local  (Extended)  Kalman  Filter). 


Figure  7.  Operations. 


Data  Fusion  Overview— Level  3  Threat  Estimation 


The  Threats  of  Concern  for  the  Use  Case  are  those  to  the  two  Special  Ops  Aircraft  shown 
in  Figure  7.  Since  they  are  observing  OpSec  and  do  not  have  sensors  that  can  detect  the 
pop-up  Threat,  it  can  be  seen  that  they  may  be  at  risk  to  that  Threat  depending  on  various 
factors.  Using  a  Threat  estimation  paradigm  that  was  used  on  another  AFOSR  project  for 
Edwards  AFB  and  considered  plausible  for  Joint  Strike  Fighter  basic  research 
applications  on  that  program,  we  modified  that  approach  for  this  effort  to  exploit 
AFOSR-funded  research.  That  approach  involves  a  logic  that  estimates  the  Actual  Risk 
to  a  friendly  aircraft  by  the  relationship  between  an  Inherent  Risk  and  the  ability  to  thwart 
that  risk  with  available  countermeasures.  The  way  this  notional  process  is  being 
implemented  is  shown  in  Figure  8. 
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Figure  8.  Concept  of  Risk  (~  Threat)  Estimation. 

The  basic  idea  is  for  the  friendly  platform  to:  (a)  sense  the  hostile  emitter  modes— here  RJ 
SIGINT,  (b)  understand  the  friendly  platform-to-hostile  threat  geometrical  aspect 
(Airborne  Intercept  Aspect,  AI  Aspect  in  the  figure)— here  this  is  done  by  the  RJ  doing  a 
track  estimate  for  the  AC- 130  SpOps  gunships  assumed  possible  via  periodic  datalink 
messages  to  the  RJ  and  an  Extended  Kalman  Filter  on  the  RJ,  and  (c)  estimate  the  Time 
to  Lethal  Envelope  (TTLE)  by  knowing  the  AC- 130  tracks  and  the  emitter  Kill  Zone. 
The  Mode  and  Aspect  provide  a  sense  of  how  much  information  the  hostile  may  have  on 
the  friendlies,  and  whether  the  friendlies  are  in  geometry  favorable  to  the  hostile— this 
provides  an  estimate  of  Hostile  Intent.  Lethality,  usually  interpreted  as  the  ability  to  do 
harm,  is  associated  with  the  TTLE  parameter.  In  turn,  Intent  and  Lethality  can  be  used  to 
develop  an  estimate  of  Inherent  Risk  (meaning  the  risk  present  with  our  consideration  of 
friendly  countermeasures).  Actual  Risk  is  then  the  Inherent  Risk  mitigated  by  the  choice 
of  a  possible  Countermeasure  and  its  effect  on  the  Threat. 

Our  approach  to  these  Fusion  calculations  involved  a  Fuzzy  Logic  (FL)  technique  used 
on  the  other  AFOSR  program,  modified  for  this  RJ  type  application  and  Use  Case.  FL 
methods  first  “fuzzify”  the  “crisp”  or  real-values  of  pertinent  parameters  and,  using 
membership  functions,  represent  a  degree  to  which  the  effects  of  that  parameter 
influences  a  consequent  result.  These  dependencies  are  expressed  in  Logic  Rules  that 
represent  the  asserted  relationships.  The  composite  effect  is  determined  by  the  applicable 
set  of  rules  for  given  parameters  and  the  final  result  determined  by  a  “defuzzification” 
step,  yielding  the  Threat  or  Inherent  Risk  value  as  a  number  in  the  range  (0,1).  These 
Threat/Risk  values  are  used  by  the  Task  Nomination  logic  and  DRM  process  to 
determine  if  and  when  the  AC- 130  platforms  should  deviate  from  their  planned  courses. 
The  overall  Fuzzy  Logic  process  flow  is  summarized  Figure  9. 
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Figure  9.  Summary  of  Fuzzy  Logic-based  Threat/Inherent  Risk  Estimation  Process. 


Results 


As  our  effort  focused  on  proof-of-concept,  no  formal  experimentation  and  analysis  was 
done  regarding  the  results,  but  clear,  quantitative  results  were  realized  that  gave  clear 
indication  of  correct  trends.  The  results  are  shown  with  screen-shots  from  our 
demonstration  spawning  from  the  Use  Case  (see  Figures  10  and  1 1). 

Figure  10  illustrates  events  just  after  the  RJ  detects  the  SIGINT  from  the  pop-up  threat, 
here  located  in  the  Yellow  region  of  the  figure;  the  title  of  the  figure  elaborates  on  the 
circumstances  shown.  Note  the  Threat  boxes  on  the  middle  right  side  of  the  screen;  no 
values  are  computed  as  yet,  since  the  AC- 130  platforms  are  well  out  of  range.  Initially 
all  3  available  UAVs  are  sent  to  develop  the  needed  precision  geolocation;  the  screen  also 
shows  a  later  moment  after  DRM  logic  has  determined  that  the  lower  2  UAVs  (Global 
Hawks)  are  sufficient  for  the  geolocation  task.  As  a  result,  the  free  UAV  is  sent  to  re¬ 
focus  on  the  second  ISR  mission. 

The  second  screen  (11)  illustrates  the  more  precise  geo-location  in  progress  using  two 
Global  Hawks,  with  Fusion  occurring  on  the  RJ.  In  the  mean  time  the  Fuzzy  Logic 
Threat/Risk  calculation  is  performed  for  the  SpOps  gunships  as  they  are  advancing 
towards  SAM.  The  threat  values  are  shown  in  two  text  boxes  in  the  right  side  of  the 
window.  Once  the  threat  becomes  relatively  high,  the  DRM  logic  directs  Gunship  #1  in  a 
new  direction  to  vector  away  from  the  kill  circle  of  the  SAM.  Change  in  direction  is 
shown  with  dashed  red  line.  DRM  logic  also  assigns  one  of  the  F-22’s  to  fly  towards  the 
SAM  to  provide  cover  for  the  SpOps  operations.  The  arrow  is  pointing  towards  geo¬ 
location  error  ellipse  (Major  axis  is  around  650  meter  and  minor  axis  is  around  500 
meter). 

In  summary,  the  composite  multi-Level  Data  Fusion  and  Dynamic  resource  Management 
Logic  operations  show  correct  trends  and  positive  results.  Precision  emitter  geolocation  is 
achieved  sufficient  to  provide  threat  avoidance  (note  the  reduction  in  error  ellipse  size 
from  10km  to  650m  semi-major  axis  size). 


14 

IAVO  Research  and  Scientific 
STTR  Final  Report:  FA9550-07-C-0085 


Legends: 
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Figure  10.  Simulation  Scenario  when  RJ  detects  hostile  emitter  signal  in  yellow  region 
and  first  Geo-location  has  been  done.  Global  Hawk  Tasks  and  the  path  have  been 
modified  as  per  task  re-nomination  logic.  The  change  Route  is  shown  in  dashed 
red  lines.  Arrows  point  towards  the  error  ellipse  (blue)  for  the  located  SAM 
(Major  axis  of  ellipse  is  around  10  kilometers).  The  black  circle  is  the  calculated 
Kill  Zone  for  the  located  SAM. 
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Figure  11.  Near-end  point  of  Use  Case  Simulation.  Global  Hawk-based  Precision 
Geolocation,  Threat  Avoidance  by  AC- 130  Gunships. 


2.2  Task  2:  Data  Acquisition 


While  our  goal  was  to  acquire  relevant,  “real”  RJ  operation  data  used  by  KAST,  we  were 
unable  to  do  so.  As  a  workaround  we  sought  sources  that  would  complement  our  needs 
relevant  to  the  stated  AFRL  RIEA  Phase  I  objectives  and  our  proposed  fmCortex™ 
solution.  These  included  data  available  from  earlier  R&D  initiatives  that  team  members 
were  involved  in  that  were  applicable  to  those  methodologies  we  were  seeking  to  modify 
and,  or  develop.  These  were  consistent  with  our  proposed  approach  -  and  relevant  to  DF 
and  DRM  within  the  context  of  inter-platform  routing  and  collaborative  flow 
management.  An  example  was  an  initiative  that  was  sponsored  by  AFFTC  Edwards  AFB 
that  focused  on  Multi-platform  Air-to-Air  EW  Engagement.  We  were  also  able  to 
leverage  platform  routing  analyses  conducted  in  like  domains  and  tailor  those 
accordingly. 

Our  baseline  approach  and  the  associated  data  considered  a  dynamic  adaptation  to  an 
existing  ATO  to  deal  with  contingencies,  pop-up  threats,  etc.,  while  simultaneously 
finding  the  ideal  means  to  reassign  mission-tasking  segments  to  the  entire  multi-platform 
suite.  Our  Use  Case  and  the  resulting  methodologies,  ontologies  and  algorithms  will  lend 
themselves  well  to  future  development  in  the  required  domain. 
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2.3 


Task  3:  Platform  Dynamic  Resource  Allocation 


For  this  task  emphasis  was  placed  on  the  cooperative  dynamic  resource  allocation 
requirement  for  the  airborne  suite  of  multiple  ISR  platforms,  addressing  the  need  for  a 
capability  to  dynamically  assign  existing  and  new  mission  tasking  segments  to  the 
various  platforms  as  both  contingencies  and  pop-up  threat  conditions  inevitably  arise. 
This  was  done  while  simultaneously  finding  the  best  way  to  reassign  mission  tasking 
segments  to  the  entire  multiplatform  suite,  using  overall  mission  effectiveness  as  its 
central  optimization  criterion.  Our  solution  is  a  recommendation  to  mission  staff.  [Note: 
we  use  the  term  “recommended”  for  both  the  task  reassignments  and  the  new  routings, 
since  we  intend  to  invoke  these  capabilities  as  decision-aids  to  appropriate  mission  staff, 
not  as  a  fully  automated  capability  that  we  consider  unrealistic  in  the  dynamic 
environments  that  are  encountered  in  today’s  operations].  As  things  are  always  dynamic, 
we  employed  a  solution  that  uses  a  (possibly-variable)  time  slice  approach  in  order  to 
deal  with  contingencies  across  the  mission-planning  horizon.  [Note  too  that  there  is  an 
important  feedback  loop  that  must  be  incorporated,  so  that  the  reassignment  logic  is 
aware  of  what  tasks  are  being  completed  or  what  other  contingencies  may  be  being 
experienced].  We  developed  a  basic  dynamic  re-tasking  model  and  provided  a  much 
more  effective  and  extensible  model  then  initially  conceived.  Initial  mission  components 
only  included  ‘time  on  target’  or  ‘snapshot’  tasks  and  continuous  (e.g.  video  surveillance) 
tasks.  The  model  is  now  extensible  to  a  very  wide  variety  of  mission  components  as 
detailed  in  the  following  section  with  geolocation  tasks  and  the  concept  of  cover  or  escort 
tasks  as  well  as  additional  constraints  on  fuel  capacity/consumption.  The  following 
section  provides  more  details  on  the  mathematical  aspects  of  the  solution. 


2.4  Task  4:  Develop  mathematical  techniques  and 
routing/cueing  algorithms 


The  use  of  mathematical  programming  (MP)  was  the  paradigm  chosen  for  optimal 
resource  allocation.  Mathematical  programming  refers  to  a  class  of  analytical  (algebraic) 
methods  that  can  find  the  best  way  to  achieve  a  given  objective  while  complying  with  a 
set  of  constraints.  MP  models  determine  the  optimal  allocation  of  resources  among 
competing  alternatives  within  an  operational  system.  In  addition  to  the  obvious  practical 
benefit  of  generating  optimal  solutions,  MP  provides  a  sound  theoretical  basis  for 
properly  understanding  the  broader  implications  inherent  in  the  framework  of  the  solution 
structure.  For  example,  the  developed  models  depict  the  consequences  of  alternative 
courses  of  action  by  quantifying  the  opportunity  costs  of  scarce  system  resources.  MP 
comprises  a  variety  of  paradigms  (theoretical  frameworks)  tailored  to  different  kinds  of 
problems,  to  include:  linear  programming  (LP),  integer  programming  (IP)  for  problems 
requiring  integer  solutions;  nonlinear  programming  (NLP)  where  the  objective  and/or  one 
or  more  constraints  are  nonlinear  functions;  and  goal  programming  (GP)  for  problems 
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with  multiple  objectives.  The  significance  of  developing  this  dynamic  resource 
allocation  capability  is  to  off-load  the  cognitive  workload  of  managing  such  issues  in  an 
optimal  way  from  the  flight  crews,  enable  much  more  rapid  response  times  to  determine 
the  reassignments  under  stressful  and  possibly  life-threatening  conditions,  while  at  the 
same  time  maintaining  a  focus  on  overall  mission  effectiveness. 

In  the  abstract,  parameters  and  estimates  provided  by  the  DF  system  are  among  the 
various  criteria  used  in  the  Task  Nomination  Logic  Matrix,  which  is  a  matrix  containing  a 
list  of  tasking  options  (typically  the  list  of  available/feasible  resources),  and  the  various 
parameters  and  metrics  associated  with  each  resource  choice.  This  matrix  is  scanned  by 
the  DRM  logic  to  assess  the  best  options  from  the  point  of  view  akin  to  solving  an 
Assignment  problem.  However,  the  DRM  logic  does  not  take  these  initial  resource 
assignments  as  final  but  weighs  them  in  a  separate,  mathematical  programming  logic  that 
formulates  a  set  of  complex  constraints  and  objective  functions  that  frame  the  mission- 
level  optimization  approach. 

In  addition  to  the  time-on-target,  snapshot  and  continuous  task  types  in  the  earliest  model 
formulation,  we  added  the  following  additional  criteria  to  the  modeling  repertoire: 

Updates  to  the  Model 

•  The  objective  function  formulation  has  been  extended  to  allow  the  user  to  provide 
preferential  time  slices.  For  example,  in  a  given  time  window,  it  might  be 
preferable  to  perform  the  task  as  early  as  possible; 

•  A  “moving  surveillance”  mission  component  type  has  been  added.  These  are 
useful  in  modeling  UAV  surveillance  where  the  area  of  interest  is  a  path  between 
two  locations,  rather  than  a  single  location.  Here,  two  locations  on  the  battlefield 
are  identified,  and  a  single  resource  is  tasked  to  perform  continuous  surveillance 
over  the  entire  path; 

•  A  “geolocation”  mission  component  type  has  been  added.  Here,  a  pair  of 
resources,  operating  in  different  locations,  is  tasked  to  monitor  a  remote  target 
whose  location  needs  to  be  precisely  determined.  The  exact  location  of  each 
resource,  as  well  as  the  exact  time  at  which  the  geolocation  task  is  to  be 
performed,  is  selected  from  a  list  of  candidate  locations/times; 

•  The  introduction  of  “fuel  capacity”  constraints  force  a  resource  to  return  to  a  base 
location  before  the  resource  runs  out  of  fuel; 

•  The  concept  of  “support”  resources  has  been  included  in  the  model.  The  support 
resources  are  capable  of  providing  coverage  for  “vulnerable”  resources,  and  are 
scheduled  to  fly  alongside  the  vulnerable  resources  whenever  they  enter/leave 
unsafe  areas; 

•  The  model  notation  was  updated  to  handle  the  new  mission  component  types; 

•  The  model  was  updated  to  allow  a  task  to  be  performed  by  multiple  resources. 
Previously  the  model  required  a  task  to  be  assigned  to  no  more  than  one  resource. 

The  full  model  is  provided  on  the  following  pages. 
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The  Full  Phase  I  Mathematical  Programming  Model  for  Inter-Platform  Re-tasking 


Notation  -  Sets 


B 

Set  of  bases  (depots). 

Br 

Set  of  bases  (depots)  that  are  available  to  resource  r. 

M 

Set  of  mission  components  (tasks).  mGM. 

UQM 

Set  of  tasks  located  in  unsafe  locations. 

R 

Set  of  resources. 

VQR 

Set  of  vulnerable  resources  that  require  support/cover  from  support 
resources. 

SnQR 

Set  of  support  resources  that  are  capable  of  providing  cover  for 
vulnerable  resource  rv  E  V. 

T 

Set  of  time  slices  making  up  the  mission  horizon.  t£ET. 

T  C  T 

Set  of  time  slices  in  which  mission  component  /«EM  may  be 
performed. 

Rm 

Set  of  resources  that  can  perform  mission  component  am  EM.  /•£/?„,. 

Mr 

Set  of  mission  components  (tasks)  capable  of  being  performed  by 
resource  r£ER.  m£EMr. 

Mrt 

Set  of  mission  components  (tasks)  capable  of  being  performed  by 
resource  r£f?  at  time  slice  /E  T.  m£EMrt. 

A m  ( r,m ) 

Set  of  tasks  (excluding  bases)  from  which  resource  r  may  travel 
directly  to  task  m. 

A  B~{r,m) 

Set  of  bases  from  which  resource  r  may  travel  directly  to  task  m .  Tasks 
are  not  included  in  this  set. 

A  ~  (r,m) 

Set  of  all  possible  nodes  from  which  resource  r  may  travel  directly  to 
node  m.  This  includes  tasks,  initial  location,  and  bases. 

Au  0) 

Initial  location  of  resource  r. 

Set  of  all  possible  nodes  to  which  resource  r  may  travel. 

Af+(r) 

Set  of  all  possible  nodes  to  which  resource  r  may  travel  at  time  t.  This 
includes  tasks  and  bases,  but  does  not  include  initial  location  unless  the 
initial  location  was  a  base. 

Notation  -  Parameters 


r\  min 

0  Srv 

0  Minimum  number  of  support  resources  that  are  required  for 
vulnerable  resource  rv £  V. 

r\  max 

0  Srv 

1  Maximum  number  of  support  resources  that  are  allowed  for 
vulnerable  resource  rv£V. 

0  Dm 

2  Number  of  time  slices  for  which  continuous  task  m  must  be 
performed. 

a  nun 

0  nm 

3  Minimum  number  of  resources  required  to  perform  task  m. 

c\  max 

0  nm 

4  Maximum  number  of  resources  allowed  to  perform  task  m. 

ab  m  =  1 

rm  IH, 
a  b 

If  ATO  has  resource  r  (initially)  assigned  to  mission  component  nib 
during  time  slice  B  and  the  previous  mission  component  was  ma. 

f  rm  TYl, 
a  b 

Minimum  number  of  time  slices  required  for  resource  r  to  transition 
from  mission  component  ma  to  mission  component  /«/,. 
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d„,mh 

Euclidean  distance  between  mission  components  ma  and  mb-  Used  in 
objective  function  term  Z3.  d  m  =  0. 

a”  a 

Pm 

Priority  of  mission  m. 

^ rm 

Effectiveness  of  resource  r  conducting  mission  in. 

Notation  -  Decision  Variables 


x  b  —  1 

Xrm  m,  1 

a  b 

If  resource  r  is  to  be  assigned  to  mission  component  /«/,  during  time 
slice  tb  and  the  previous  mission  component  was  ma. 

y>n 

Number  of  'infinite'  resources  assigned  to  perform  mission-critical 
snapshot  task  m. 

t 

Zm 

Number  of  'infinite'  resources  assigned  to  perform  mission-critical 
continuous  task  m  at  time  slice  t. 

Cab 

Number  of '  infinite'  resources  required  to  provide  cover  for  vulnerable 
resources  on  the  link  between  tasks  a  and  b,  where  at  least  one  of  these 
tasks  is  in  an  unsafe  location,  (J. 

Objective  Functions 

z"  L. 


p  e 

r  rm. 


tu 


^  7/;/,  I  &mb  ^ h  DV'm  171, 

&^r,mb) 


b  ’■*  mb  'b  "'a'  “  V  b  J 

Maximize  mission  priority  and  resource  effectiveness  while  considering 
preferences  for  assigning  tasks  to  target  time  slices. 

7  \ 


^2.-- 


2  2 


1- 


a 


nnffl, 
a  b 


b  m. 


X, 


rmJfl, 
a  b 


b  m. 


Impose  a  penalty  if  snapshot  tasks  are  re-assigned  to  a  different  resource. 


7 

V  - 


l  l 

''rE\  “r,mh)tb 


1 

h.ah  ) 

\xb 

\Xm  m, 

)  a  b 

T 

\mb 

\  / 

mbmci  V 

Impose  a  penalty  if  video  tasks  are  re-assigned  to  a  different  resource. 


z*--  y 


v 


1 bl 


jnu 


bl  m. 


1  —  a  b2  ^  ^ h 3 


rm Wlfo  Jrma^b 


*b3  *  tb2 


Impose  a  penalty  for  changing  the  assigned  completion  time  for  snapshot 
tasks. 
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Z3-  X 

mb^M'<3Tnh  m<7GA ~[r,mh) 


d. 


mu 


x„: 


,rriu 


%-a - -  V  ’  b  ' 

*  m. 


b  m. 


Minimize  the  distance  traveled  for  each  resource. 


z4-- 


2  2^-22/ 

mec  matter  meSter 


Minimize  the  use  of  “infinite  resources”. 


Computations  for  scaling  objective  function: 

z,r.k(:w,j 

Z2a  +  Z2b  £  H  Z2c  *  KCr  U  MC  U  NMC\ 

If  Z2  =  Z2a  +  Z2b  or  if  Z2  =  a(z2a  +  Z2b )+  (l  -  d)Z2c ,  then  Z2  <  Z2'“  -  \m\  where  0  <  a  <  1 . 


z3  <  z3mflX  -  \m 


i  /  max 


jn. 


^maCM,mbCM  a  ^ 


Some  possible  objective  function  formulations  include: 

,  z,  /  , \Z2  +z,h  „  z,  „ 

Max  a  — —  +  (l  -  a  )  2  — —  +  a  — —  +  aZ, , 

y  max  x  7  7 max  7 max  ^ 

z^i  z,2  z,3 


Max  d- 


max 


-  +  (l-dXZ2a  +Z2b)  + 


max 


r,  m 


v  e 

Jr  m  n 


d„ 


+  diZ 4 ,  or 


m  G M, mh  £M  '"aWfe 


A/r  '  Zi  ,  A  '\d(Z2  +z2b  )+(l  ~  d)z 
Max  a  — —  +  (1  -  a )  - — — 2 - n 

'-7  max  x  '  t  max 


z 

2c  +  d  — —  +  dZ, , 


where 0  <  b  <  1 ,0  <  a<l,0<a<l, and d : 


/  max  ' 

^  m  G  M 

* 

\ 

/  max 
m  G  M,  r  G  R 

’  n 


Type  I  Constraints 

Type  I  constraints  describe  requirements  for  perfonning  a  given  task. 

MCA  Tasks 

MCA  tasks  are  mission-critical  tasks  to  which  resources  must  be  assigned  at  any 
time  in  Tm. 
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max 


r  AW  tb^rmb  ma<EA  ( r,mh  ) 


Xln,  m  +ym  -  nm  mdX  Vaw,  GMC4 

rm  I'Lfo  ^  aw  >  mb  & 


rE^w  tb^mh  maEA  (r,mb) 


2  2  x‘k»hsl  Vm,  €E {MCA  2), r£^, 

ma£A  (r,m6) 

MCAL  Tasks 

MCAL  tasks  require  that  a  resource  will  be  busy  in  transit  from  task  mb  to  task  mc. 
MCAL  tasks  may  be  used  to  model  a  video  surveillance  task  in  which  video  is 
captured  over  a  path  (rather  than  at  a  single  location). 

2 

rE(R  D 

v  mb 

2 

r&R  D 

v  mb 

2 

n 
mh 

2 

mb n 

2  * 

tcETmc 

MCG  Tasks 

MCG  tasks  are  mission-critical  geolocation  tasks  in  which  exactly  two  resources 
are  dispatched  to  two  different  locations  at  the  same  time. 


V  xj’  m  +y  —  n,  Vm.EMCA 

/ j  rm  til,.  J  mi.  awa  b 


mh  m..  '  b  mb 


mh  m..  ’  b  mb 


2 

xh 

rm 

c 

,mb  + 

yn,b 

max 

<  n 

mb 

V  ( mh ,  mc )  E  MCAL 

maEA  ( r , 

mb > 

2 

Jb 

rm^ 

,mb  + 

y,nb 

min 

>  n 

mb 

V  ( mh  ,/h)G  MCAL 

maEA  (a*, 

« >b ) 

Jc 

X,-m,  m 

b  c 

+  V 

'  "V 

-  n>nc 

max 

V(mb,mc 

)  G  MCA  L 

Jc 

Xrmhmc 

+ym 

c 

~  n’»c 

min 

V(mh,mc 

)  G  MCA  L 

2  2 

ma (EA  (r,mb)  tcETmc 


xL  m  V (mb ,  mc )  G  MCAL,  r  G  (R  Pi  Rm  ) 


^mbmc 


NMC  Tasks 

NMC  tasks  are  not  required,  but  may  be  useful/beneficial. 

'rm  TYl , 


2  X^m  <1  V  m ,  e  NMC 


r^m  tb^mb  maEA  (r,mh ) 

CMC  Tasks 

CMC  tasks  are  mission-critical  tasks  that  must  be  performed  over  a  continuous  set 
of  time  slices. 


x'l  m  +  z*  <  D  Vm,  G CMC 

rm  frli  mb  mb  b 


r^m  tb^mb  maEA  (r,mh ) 
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CNC  Tasks 

Continuous  tasks  that  are  not  critical  but  may  be  useful. 


V/«,  <ECNC,t.<ETm 

b  ’  b 


Type  II  Constraints 

Type  II  constraints  ensure  that  each  resource  is  assigned  to  no  more  than  one  task  at  a 
time,  according  to  its  capabilities. 

Y  2  x™amb  - 1  V''eR,»Gr  (2.1) 

»‘b^th(r)maEA(r,mb) 

Type  III  Constraints 

Type  III  constraints  represent  time/space  constraints.  Let Xm|,„2  represent  the  minimum 

duration  (number  of  time  slices)  required  for  resource  r  to  move  from  mission  component 
m i  to  mission  component  mi.  Each  resource  requires  a  single  “minimum  duration  table”, 
resulting  in  |R|  tables.  Each  table  is  a  square  matrix  containing  (\M,\  +  1)  rows/columns; 
\Mr\  mission  components  for  which  resource  r  is  capable  plus  one  “dummy”  mission 
indicating  the  initial  location  of  resource  r. 

The  following  logic  is  used  to  construct  a  set  of  constraints  for  infeasible  mission  pairs: 
foreach  resource  r  G  R  { 

foreach  mission  component  mc  G  A+  (r)  { 

foreach  time  sliced  G  Tm  { 

foreach  mission  component  mb  G  { A“  (r,  mc )  \  A0  (r ) :  mb  *  mc }  { 
foreach  {tb  GJ^  :tb  <tc  }U  t0  { 
d  tc  ~  tb 

tifrmbmc  >  d  I 

T  Xnna,nb  C3'1) 

maGAu  ( r,mb ) 


Type  IV  Constraints 

Type  IV  constraints  specify  the  network  structure. 

•  Each  task  must  be  linked  to  a  predecessor  task. 

Vr  G  R,  mc  G  A+  (r), 


maE. A  ( r,mb ) 


Xnn  m.  ~  Xnn,m 
a  b  be 


th  +  f 

h  J  rr. 


b"‘c 


<  t„ 


mh  G  {A  ( r,mc )  \  (A0  (r)  PiAm  ( r,mc ))}, 
t  Gf 

c  m 

c 


•  Each  snapshot  task  may  have  no  more  than  1  successor  tasks  for  a  given 
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resource. 


2  y  W£B,»,e(M7UA°(r)) 

ffii,e{A+  (r):ma 6A  (r,m4  )}  tbeTmb 


•  Do  not  allow  resource  to  “split”  on  continuous  tasks. 

y  2<-  *  2  Vre*,«.e*rV“er 


rmamh  rmhmr 

h^Tmb  '■  tb  si1™”  }m„EA"  (r,m4  )  mb£A*(r,mb ) 


•  Do  not  allow  resource  to  “split”  at  a  base  location. 

y  2<*»  2  y.i.„,  Vre«,*.e«r.<“er 


h^iTmb‘-th^tmax}maG A  (r,m6) 


mhE.A+  (r,mb)  tc ^mc 


If  a  resource  is  used,  it  must  start  from  its  initial  location. 

s  2  2  A,.,..  a  2  2  ' 


™*£{A  (r):A  (r )EA  (r,mb)}  tb^Tmh 


X%Ur},nb  S  2  2  2*X»c  VrGi? 

mc(EA+  (r)  mh(EA  ( r.mc )  tcETmc 


Note  that  f?  is  a  “big”  number.  B=\M\\T\  will  suffice. 


2.5  Task  5:  Develop  optimal  Data  Fusion  approaches, 

METHODOLOGIES 


As  detailed  and  depicted  in  the  Use  Case  under  Task  1,  consideration  was  that  fusion 
took  place  “on”  the  RJ,  and  involved  algorithms  that  estimate  fusion  based  threat  and  risk 
estimation  derived  from: 

•  A  Fuzzy  Logic  Risk  calculation; 

•  RJ  and  multi-UAV,  fusion-based  Geolocation  algorithms. 

The  Use  Case  involved  a  modern-day  mission  concept  where  ISR  assets  indirectly 
supported  an  airborne  Special  Operations  mission.  In  the  scenario  was  critical  that  the 
location  of  a  threat  air  defense  emitter  be  precisely  geo-located  in  order  to  provide 
assured  safety  to  the  airborne  SOF  platforms. 

As  a  result,  the  focus  was  on  providing  two  approaches  to  geolocation,  one  involving  a 
Kalman  Filter-based  method  and  the  other  an  angle/range-based  method.  In  addition,  we 
employed,  a  Fuzzy  Logic-based  threat  estimation  logic  that  employed  the  kinematic  and 
geometric  estimates  provided  by  the  geolocation  calculations  to  estimate  the  risk  to  the 
SOF  platforms  (see  Figure  12).  This  risk  calculation  was  employed  by  the  dynamic 
resource  allocation  logic  to  optimize  the  reassignment  of  ISR  assets  for  SOF  protection. 
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Fuzzy  Logic  Fused  State  Estimator 


[Mode  State  :  Search  vs  Track  vs  Lock-On 


•Date  Time 
•Latitude 
•Longitude 
•Error  Ellipse 
•Ellipse  Orientation 
•ELNOT 
•Frequency 
•MODE 


FIXED 

(previously 

seen) 


MOVED 


NEW 


Everything  matches  but  Not  previously  seen 
Location 


Mission  Context 


For  each  Emitter: 

ISR  Alert  1  (p) 
ISR  Alert  2  (p) 
ISR  Alert  3  (p) 


Figure  12.  Notional  Fused  State  Emitter  Estimate. 


Thus,  consistent  with  the  design  philosophy  described  above,  what  would  be  develop  is  a 
cooperative  ISR,  closed-loop  process  involving  two  fusion-based  functions  (geolocation 
and  threat  estimation)  and  a  robust  dynamic  platform  and  functional  re-tasking  logic  that 
is  based  on  optimization  techniques  employing  mathematical  programming  (see  Figure 
13). 
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Figure  13.  ISR  Alert  to  Tasking  Logic 
(Note:  this  is  new  and  needs  to  be  developed). 


2.6  Task  6:  Define  and  develop  the  flow  management  core 

ARCHITECTURE  (fm Cortex™) 


While  were  unable  to  gain  access  to  any  of  the  metadata  used  by  the  KAST  software 
relative  to  the  RJ  -  which  would  have  enabled  us  to  make  even  further  progress,  we  were 
able  to  define  (and  refine)  an  extensible  architecture  in  the  context  of  what  was  initially 
proposed  (Note:  emphasis  is  on  the  word  refine  in  that  what  is  shown  in  Figure  XX,  is  a 
more  tailored  framework  based  on  the  DF  and  DRM  capabilities  developed).  The  core 
architecture  represents  those  functions  described  prior  and  would  be  employed  as 
flexible,  extensible  architecture  with  the  ability  to  combine  processing  and  parameter 
settings  on  the  fly,  predict  and  examine  the  quality  of  the  results,  and  estimate  the 
processing  time  and  resources  required  for  the  intended  processes.  With  those 
requirements  in  mind,  several  important  design  objectives  became  apparent,  including: 

Easy  to  add  new  tasking,  logic,  and  processes; 

Support  multiple  levels  of  processing  for  each  task; 

Automated  as  possible; 

Support  multiple  data  sources; 

Easy  to  add  support  for  new  data  sources; 

Management  of  large  numbers  of  data  sources; 

Self-describing  data; 

Audit  trails  of  the  processing  done  to  the  data;  and, 

Attention  to  distributed  processing  objectives. 
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Collectively,  the  fmCortex™  is  comprised  of  those  fusion  and  dynamic  resource 
functions  created  for  or  used  in  the  context  of  our  solution.  These  form  the  basis  of  a 
core  capability  that  when  brought  into  the  fmCortex™  architecture  yield  results 
consistent  with  AFOSR  and  RIEA  needs.  The  following  (Figure  14)  details  the 
fmCortex™  architecture,  individual  components  and  processes.  The  components  and 
processes  are  described  in  the  subsections  that  follow. 


Dynamic 

Resource  Allocation 
(Routing) 


Figure  14  .fmCortex™  Core  Architecture. 


2.6.1  Ingestor 

The  Ingestor  performs  the  necessary  task  of  bringing  the  data  from  the  bus  and  preparing 
it  for  use  in  the  fmCortex™  system.  The  input  source  data  must  be  carefully  examined 
and  tagged  in  order  to  be  processed  efficiently  by  the  Fusion,  Routing,  and  Analytical 
Engine  modules.  Information  retrieved  for  tagging  purposes  includes  (but  is  not  limited 
to):  geolocation,  error  models,  time-tags,  INT-ID,  threat  ID,  etc.  In  addition,  incoming 
data  must  be  validated  with  existing  accepted  data  and  models  to  determine  its 
consistency  and/or  applicability  with  known  information.  This  Ingestor  module  will 
perform  the  necessary  cataloguing  and  metadata-tagging  to  extract  the  relevant 
information  that  can  be  passed  along  with  the  raw  data  to  subsequent  processing  stages  of 
fmCortex™ . 
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2.6.2  Analytical  Engine  (AE)  Service 

The  AE  service  acts  as  the  central  distribution  hub  and  decision  making  center  for  the 
fmCortex™  architecture.  It  handles  the  data  and  disseminates  it  to  the  Fusion  and 
Routing  modules  for  additional  processing.  Central  to  the  Analytical  Engine  is  the 
concept  of  polling  stations.  Both  the  Fusion  and  Routing  modules  have  polling  station 
access  into  the  Analytical  Engine.  These  polling  stations  alert  the  modules  when  new 
data  is  ready  to  be  processed  and  carry  processed  data  back  to  the  Analytical  Engine. 
Further  discussion  on  the  functionality  of  these  polling  stations  will  be  given  in  the  next 
section.  The  Analytical  Engine  also  contains  the  core  analytical  capabilities  of 
fmCortex™,  the  logic  of  which  drives  the  system.  This  “brain”  of  the  system  is  intended 
to  be  both  flexible  and  extensible,  allowing  interpretation  of  a  variety  of  inputs, 
addition/deletion  of  input  sources  on  the  fly,  and  extensible  to  provide  for  future 
enhancements  to  the  logic  system. 

2.6.3  Validator 

The  Validator  consists  of  a  combination  of  automated  and  semi-automated  methods  to 
evaluate  the  results  from  the  Fusion  and  Analytical  Engine  modules.  The  validation 
process  is  founded  on  the  triple  concepts  of  accuracy,  consistency,  and  usability. 
Validation  of  decisions  and  fusion  results  are  both  data-  and  task-based.  Data  based 
validation  is  intended  to  answer  the  questions:  “is  the  data/model  consistent  with  past 
models”  and  “is  the  data/model  accurate”.  Task  based  validation  is  centered  on  the 
question  “is  the  data/model  sufficient  to  proceed  with  the  designated  task”.  Validation  is 
used  as  a  control  in  the  analytical  process.  The  control  is  as  follows:  are  the  intelligence 
requirements  for  the  given  task  sufficient?  If  “yes”,  proceed  with  the  given  task.  If  “no”, 
proceed  to  data  request  (e.g.,  request  to  the  Router  for  tasking  a  data  collection  platform 
to  fill  in  the  gaps  in  our  knowledge). 

The  overarching  objective  to  our  approach  is  to  employ  a  Service  Oriented  Architecture 
(SOA)  based  approach  by  enabling  system-level  architects  to  describe  functionality  in 
multiple  processor  systems  as  if  it  were  implemented  on  a  single  processor.  This 
provides  greater  architectural  flexibility,  especially  since  functionality  is  never  locked 
into  a  particular  implementation.  The  SOA  approach  was  developed  to  offer  a  common 
communication  framework  between  distributed  software  components  in  a  system  -  in  our 
case,  fmCortex™ .  SOA  architectures  allow  one  to  abstract  the  processing  functions, 
the  data  server,  and  the  interfaces  used  to  communicate  between  them,  enabling  a 
solution  that  is  both  modular,  and  readily  extensible  without  need  for  significant 
architecture  changes  between  software  updates. 
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2.7 


Task  7:  Phase  II  prototyping  design 


After  completing  each  of  the  tasks  leading  up  to  the  prototype  design  it  was  determined 
to  modify  the  original  design  of  fmCortex™  to  produce  an  even  better  streamlined, 
modular,  and  extensible  product  as  illustrated  in  the  previous  figure.  The  functionalities 
similar  to  those  presented  in  the  original  proposal,  however,  as  greater  functionality  was 
developed  under  the  Phase  I  than  what  was  originally  proposed  (and  anticipated),  coupled 
with  an  emphasis  on  an  inter-platform  approach,  we  feel  that  any  minor  modifications  are 
enhancements  -  as  well  as  being  more  streamlined  for  internal  efficiency. 


2.8  Task  8:  Market  study  ( Option ) 

Although  the  option  was  not  exercised,  (we  were  scheduled  to  begin  our  market  study 
during  month  7  of  the  base  contract)  our  objective,  and  thus  our  approach  throughout  the 
Phase  1 ,  was  to  lay  the  foundation  for  a  capability  that  would  initially  lend  itself  to  the 
AFRL  RIEA  CMS/KAST  program.  Additionally,  our  underlying  approach  throughout 
the  development  process  is  to  continuously  maintain  consideration  for  marketing 
opportunities.  What  follows  are  just  a  few  of  the  potential  markets  (by  function)  where 
an  fmCortex™  solution  could  be  brought  to  bear: 

•  Supply  chain:  Defense  (to  include  a  myriad  of  contractors),  Transportation, 
Vendors  (e.g.,  E2open,  Boeing,  Hitachi,  IBM,  LG,  Motorola,  Exostar,  BAE, 
Lockheed  Martin,  Raytheon,  etc.)  (there  are  many  others); 

•  Logistics:  Aerospace  and  Defense,  Transportation,  Cargo  Owners, 
Manufacturing,  Retail; 

•  Multi-sensor  and  Data  fusion:  Aerospace  and  Defense  (to  include  a  myriad  of 
contractors),  DHS,  Law  Enforcement,  Data  Warehousing,  Medical,  etc.  (again 
there  are  many  others); 

•  Dynamic  Resource  Management:  Aerospace  and  Defense  (to  include  a  myriad  of 
contractors),  Pharmaceutical,  Medical,  Application  Developers  (e.g.,  Planisware, 
Serena  Mariner,  etc.); 

•  Collaborative  Flow:  Aerospace  and  Defense,  FAA,  Pharmaceutical,  Medical, 
Application  Vendors  (e.g.,  3G  Interactive,  Fujitsu  Software  Corp,  IBM, 
Imagesoft,  Taligent,  etc.). 

This  large  customer  base  continues  to  reinforce  our  determination  for  pursuing  this 
transition  path.  While  our  list  includes  potential  competitors,  e.g.,  Defense  contractors, 
we  foresee  great  potential  in  either  integrating  the  fmCortex™  product  into  their 
environment  or  them  purchasing  the  software  to  perform  the  integration. 
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3  Conclusions 


We  have  accomplished  each  of  our  Phase  I  tasks,  exceeding  our  original  goals,  and  are 
well  suited  to  begin  Phase  II  development. 

Our  approach  was  the  development  of  a  Data  Fusion,  Dynamic  Resource  Management, 
and  Platform  Routing  (PR)  within  a  Collaborative  Flow  Management  system  having 
multiple  capabilities,  each  converging  toward  the  collective  advanced  support  of  the 
Rivet  Joint  -  through  the  CMS/KAST  initiative.  We  described  in  detail  the  tools  required 
to  lay  the  foundation  of  the  fmCortex™  solution  including:  pre-  and  post-processing 
issues  critical  for  successful  fusion,  validation,  data  fusion  in  the  context  of  cooperative 
ISR,  standard-product  assessment,  entity/event/alert  generation  and  re-generation, 
dynamic  tasking,  etc.,  as  well  as  dynamic  database  components. 

Our  Phase  I  efforts  in  DF,  DRM  and  PR,  focused  on  a  mix  of  these  categories  of 
applications,  enabling  us  to  explore  and  develop  proof-of-concept  technical  capabilities 
addressing  exemplar  problems  in  reflecting  the  need  to  balance  resource  demands  across 
composite  mission  operations. 

The  major  conclusions  from  our  work: 


•  A  layered  Level  1  Precision  Geolocation  framework  and  algorithm-set  was 
prototyped  and  successfully  demonstrated; 

•  A  Fuzzy  Logic  Level  3  Threat  Estimation  fusion  logic  was  prototyped  and 
successfully  demonstrated; 

•  Preliminary  Task  Nomination  Logic  was  developed  and  integrated  into  the  adaptive 
DF-DRM  process; 

•  A  robust,  computationally-efficient  Mathematical  Programming  approach  to  DRM 
was  designed,  modified,  and  successfully  demonstrated; 

•  These  technology  prototypes  have  shown  good  promise  for  Cooperative  ISR  through 
a  synergistic  interoperation  between  Data  Fusion  and  Dynamic  Resource 
Management  quantitative  techniques; 

•  An  open,  modular,  extensible,  and  more  efficient  architecture  was  designed  and  is 
poised  for  future  development. 
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